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SECTION I . . ^ 

' ' . ■« ' ' ' 

>V * ■ , INTBOOUCTION 

J^^'t • if* 

This report discSsses developments in computer ba^ed instruction (CBI) 
and presents initiatives for the improvement of Navy instructional management 
in the 1985 to 1995 time fratfle. During the past decade computer based instruc- 
,tion has been widely applied in the U.S. Navy. The Navy's Computer Managed 
Instrufction (CMI) System, which became operatidnal in 1973» provides a means 
for guiding students through a cyrriculum of instruction. presently serves 
14 schools at five geographic locations. The approximate student average' on 
board (AOB) and throughput for CMI/courses in FY 78 were 7»000 and 65\000» 
respectively. A plan/exists to expand this system to 25 schools'and an AOB of 
16»(M)0 students at six locations. Computer, managed instruction is ^the largest 
component of CCI in the Naval Education and Training Command (Zajkowski, " 
Heidt» Core^» Mew, and Micheli, 1979). ' , 

■ fs... ^, ' > ' * • 

The use of^ccHnputers In the managenient of in$truct1on also has been 
instrumental in enhancing the Navy's capability for the delivery of individ- 
^ualized instructions. In fact» the extent to which instruction has been 
individualize '*... would have been impossible for the number of students vh 
are dealing with were it not for the *avai lability of CMI.... It gives a partial 
answer to the restraints the Congress is placing upon us for reductions in the 
costs of training and the .size of training staffs" j[Scanland» 1975)* 

All the military- services intend, to increase their use of CMI. Table 1 
shows the projections for -military CMI systems planned for the early 1980s 
(Van Matre, 1979), ^ . 
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TABLE 1 


. MILITARY 


CMI SYSTEMS 










NAVY 
• CHI 


AIR FORCE 
AIS 


ARMY 

CTS 


AIMS 


' > MARINE CORPS 
CBE. 




students Daily 


16.000 


■J 

"2,400 


365 - 


1.600 


, ^ 2,000 




Courses /Schools 


.25 


4 


- . 4 


• 2 


4-8 CAI 
40 + CMI 




locations 


6 


" . ^ 1 - 


1 


22 


. • 1 ■ 



Source: Van Matre (1979) ' ^ s 

At the November 1979 meeting of *he Chief of Naval Education and Training 
(CNEt) Experimental Training Programs Policy Board it'was pointed out that 
due Jto technological advances, the current Navy CMI system will become oXitdated 
in the 1985 to 1995 time frame. The CNET tasked the Training Analysis and 
Evaluation Group (TAEG) tfto conduct a' study of the current anit projected 
state-of-the-art' in the technology associated "with the management and/or 

1 

delivery of instruction as a basis for developing initiatives for the future. 

PURPOSE OF THE STUDY y 

The present stuc(y {!) assesses the state-of-the-art la iastructioaal 
maaagemeat aad iastruetior^l ctelivery» (2) establishes projectioas of the^ 
capabilities for. iastructioaal maaagemeat aad iastructional delivery systems . 
during thtf>1985 to 1995 time frame» and (3) ideatifies viable aUeraativ,es to 
the curreat Navy CHI system. . \ 

APPROACH ' ^ , ' . 



There were five major ccwnpoaents to the approach used ia this study. 
Firsts ar^ aaalysis was made of curreat Navy CBI systems .ia terms o, capabili- 
ties and problems. Sec6ad» literature dealing with curreat and future treads 
of instructioaal managemeat aad delivery was reviewed. Third, visits were 



'1 



CNET Itr Code N-5 of 19 Nov 1979 



S 



ERIC •., 



TAE6 Report -No. 89 



mie to acknowledge^ experts {listed it» appetidiXvA) for discussion and assess- 
ment of Navy needs and projected technologic^/ capabilities in hardware, 
•software, and courseware to satisfy these needs. Fourth, functional coInponentr^ 
of computer-based instructional mariggetnent systaj^ were identified and analyzed 
In order to arrt^ at the- functions 'of an ideal instructional management 
systan- Finally, alternative systems capSble of providing the functions were 
identified and assessed for implantation constraints which would impact on 
operational use during the- 1985 to 1995 time frame. * 

ORGANIZATION OF THIS REPORT , * ' ■ 

/ 

In addition to this introductory section, this report cohtai^ns five 
sections and two appendices. Section II describes the major categories of 
SBU discusses trends in GBI, and provides an economic projection of CBI 
developments^^ Section Ilt proyides a discussion of instructional management 
systems- Section IV provides the state-of-tfie-art and trends of instructioni^l 
delivery- systems.' " Section V presents four CBI alternatives and contains an 
-assessment oiF these alternatives in terms of key system characteristics. 
Section VI provides recomnendations for actions to be taken in support, df 
1985 to 1995 ^I system development. Appendix A lists the Gpttsultants con- . 
tacted during %his study. Appendix B describes* the major categories, sub^ 
categories, and functions of CBI- , 
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SECTION II 



CATEGORIES, TRENDS, AND ECONOMIC ASSESSMENT - 
*«- DP COMPUTER BASEO INSTRUCTION 

This section (I) identifies 'arrd defines "the major categories of CBI to 
be analyzed" in 'the remainder of fhe report, (2) discusses trends in CBI, and 
(3) provides economic projections of CBI developments. Computer based instruc-, 
tion a generic term used to describe ahy-lnstrtictlonal system In which 
computer capablHtlesv are utilize^ to manage, deliver or support Instruction,^ 
or to maintain administrative information onMnstructiohal processes. .This 
definition does not provide sufficient specifi^cfty for detailed comparative 
analyses of various systems. However* a general industry acceptance ot two. 
categories of CBI has evolved* computer assisted/aided instruction '(CAI) and 
computer managed instruction (CMI). The former is invplved in the det^very 
of instruction* while the latter, in general* simply manages a program of 
instruction without a delivery component* Even though these functions are 
distinctly different, the fact that th^ can be employed together in a single 
system has given rise to some coVusion in terms. Therefore* an attempt is 

made in this report to provide a framework for under staTiding and 'differentiating 

it 

between categories of CBI. , • . 

MAaOR CATEGORIES OF CBI- , ' > 

Major categories of CBI to be addressed in this study are shown in 
figure 1. Computer-based instructional management (CBIM) includes those 
functions which manage o^^uide a student through an instructional sequence 
such as testing* response analysis* nrescriptive, guidance, and student mon- 
itoring. Computer-based instructionaKadministration (CBIA) inclines all -of 
the record (teepihg and repoHing required for effective program adninistratton. 
Computer based instruction support (CBIS) includes instructional/l^drnlng ^ 
respurc^ management and allocation* program development coordination* config- 
uration managemenjt* and computer support functions. The functions included 
in each of these major CBI subcategories are shown^in figure 2. They are 
also'described in greater detail in appendix B. 
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Figure 2. Major Subcategories of .CBI 
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r Computer based instructional delivery (UiO) is a unique category which 
Includes all computer based media options. These options would include 
classical CAI; computer graphics, computer controlled video, conflputer sinflula* 
tion, and computer based training devices. Computer based instructional 
delivery together with CBIM has almost unlimited potential for development 
snd^growth and will therefore be the areas in which Navy instruction will 
make its greatest advances. However, it should be recognized that the admin- 
istration and suppot^t categories of tBI (CBIA and CBIS) wilf^be essential for 
the, orderly and. cost-effective implementation of large scale. CBI systems. 

TRENDS . 

During the next decade computer developments which support, instructional 
processes are likely to evolve to the "extent that all functions which can be 
computerized will be. Therefore, a key decision at this time involves the 
choice between a system configuration which woiild result from evolutionary 
growth of existing systems and a large scale integrated system developed 
under an RDT&E progi^n in terms of m^imuin effectiveness/efficiency in the 
1985-1^95 time frame. However, a general consensus* seems to be that whatever 
general system is selected the underlying ganeral development strategy should 
include the central management of system and network standards with autonomous 
decentralized processing and data base control. This general strategy permits 
a w,ide range of local hardware/software options for.CBI while requiring 
strict adherence to central management policy relating to network configuration, 
data base management, comAuni cations protocol, and configuration management. 

" A number of initiatives will be^ required to implement this approach 
among which will be the phasing otit of the existing Navy CMI System and a 
phasing In of a new CBI system with greatly expanded capability, A major 
contributing factor to this situation is an increasing trend toward the use 
of unique computer training applicatiorts arising out of special training 
requirements such as electronic warfare training and nuclear power training. 
Since It Is relatively certain that this trend will continue, the instruc- 
tional management' network of the future must allow for inclusion of these ^ 
unique applications within, a global management structure. This can only be 
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accomplished through centrally controlled policy deve1o|Hnent and the estab- 
lis^imerit of explicit standards for computer based iastructional network 
membership; ' — ^ 

Computer technology has developed to such a poiat that computer hardware 
cost tias become a secondary consideration to computer software cost* The 
following, paragraphs address .the underlying economic considerations which 

have brought about this change. 

■ ^ ■ * 

ECONOMIC ASSESSMENT ..OF CBI ■ 

The feasibility of adopting new improvements in computer technology for 
computer based training .applications depends upon both technical effective- 
ness^and economic. efficiency. The'te|hnical -effectiveness, along with some 
, insights iato the future course of develQpmerits in instructional technology, 
is addressed elsewhere, in this repffitht. /New and improved computer technology 
.which vrilT support effective training' is a necessary but not sufficient 
condition for adopting this-ifiew technology for Naval training. The adoption 
of any improved technology must^lso be economically effici€5nt; i.e., it 'must 
also provide the most economic means of reaching the training objectives if , 
it IS to be rationally integrated into the Navy training system* Economic 
analysis- is an analytical proceduiw for determining the cost-effectiveness of 
alternatives for satisfying training objectives. " 

^s previously mentioned, this study focuses on the anticipated state of 
computer technology as it will exist for the period 1985-1995 and is^designed 
to provide policy guidance as to the appropriate planning decisions which 
wil] enable the Navy to plan for thg most cost-effective, long-range training 
system. Since the study does not address any specific training system alter* 
native^i^ it is inappropriate to provide an economic analysis supporting or' 
recommending a specific long -range elan for either accepting or rejecting 
projected computer technology. Specific recpmmendations will depend on the 
new (and as yet undefined) hardware systems, software requirements, future 
training requirements, existing resource configurations, and relative, resource 
prices.* * ^ 
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TRENDS m COMPUTER PROCESSING COSTS, One of the most important considerations 
in determinifig the economic desirability of adopting new and improved computer- 
^ technology is the relative productivity per dollar spent on both the old and 
new system. Any training task can be accomplished with an almost infinite 
set of resource combinations. Which set is the most economically efficient 
is determined by both factor productivity and factor cost. Any disproportionate 
change in either prcductivity or cost will ultimately. require a change 1n the 
relative ratios with which the resources are used in training/ Often changes 
in productivity and costs are brought about by technological advances. In , 
computer technology there has. been a substantial increase in productivity and, 
both "total cost of hardware and "average processing costs have dropped pre- 
cipitously^ These changes make H imperative that the. training system be 
reconfigured in the long run to' more fully utilize and capture the .efficien- 
cies of the new computer technology. 

The trend in productivity for hardware has shown phenomenal improvement 
over the last several years, 'Host experts believe that Jthis rate of improve- 
ment will continue through 1990. If such improvements- do continue, then 
processing'speed in 1990 is expected to be approximately 200 times faster 
than that which existed in 1970. This trend is illustrated in figure 3. 

The processing speed and anticipated reduction in real costs of micro- 
processing components will contribute significantly to more cost-effective 
computational facilities. Trendy ir\ costs of microprocessor chips are illus- 
trated' tn^ figure 4. It is expected that the cost of a 16 bit microprocessor 
chip will cost less than $10 by 1990. The cost of a 4 bit microprocessor 
chip' will be less than $1.'' ^ . ^ ' ' ^ . ^ ' 

The cost of memory is also going down. If present trends continue, as . 
they are expected to do, the 1990 memory costs per bit will be less than 1 
percent of the, 1970 costs. This trend is illustrated in figure 5. The net. 
result of improvements in microprocessor. costs, processing speed, and memory 
costs has resulted in an extremely rapid decline in average storage and 
processing costs. Projections through 1990 indicate that the average proces- 
sing costs will be less than 1 percent of the 1970 processing costs, as illus- 
trated tin figure 6. I - 
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Figure 3, Index of Trend in Processing Speed Between 
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Figure 4. ProiJected Average Costs of Microcomputer Chips 
( 1970 - 1990} ■ V ■ 

/ 

Source: Motorola Semiconductor Group (l9ftO) * " " , . 
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During the same period (in \vhich the productivity of computer technology 
was increasing and costs were dropping), the average labor costs in training 
have been increasing. Thi? change in relative price ratios shifts the economic 
advantage to those" alternative training systems which utilizp computers to 
substitute for the relatively expensive labor. 

SC/y_E ECONOMIES. Scale economies occur when average training costs decrease 
with increases in the number of students trained or size of the training 
systems. These economies can be attributed, in part, to the opportunity to -s 
use capital which is highly efficient. Until recently, it was necessary to 
use large computer hardware system.^ in order to obtain the computational 
capacity and speed necessary for trainir\g. systQms, It has simply not been 
economi;.^lly feasible to develop a computerized, training system fpr many of 
the smaller training systems. XouV^as and/cr training systems with large 
throughput were the only ones which could obtain the necessary efficiencies 
. to justify a computerized training system. The new developments in computer 

technology have changed the cost functions in ^uch a way that it will no . 
^ longer be necessary to hava a large throughput in order to capture the scaje 
economies. The importance of scale of operations or the size of the training 
systems Is becoming less of an issue, in the cost effective calculation. 

SOFTWARE DEVELOPMENT, ' The rapid increase in the technical ^capability of the - 

\ - 

' hardware has provided the opportunity to use the computer fo? more sophisticated 
* ^ -A. 

and widespread app'licatians. However, these uses often require the development 

of unique softv/are packages. Software development Is extremely labor-intensive 
> and, therefore, very costly. The increased use of hardware will require 
greater commitment of resources to software development,, and in the lonjg run 
software costs will.be the dominant cost factor in the selection of a com- 
puterized training system. This^ increased cost can be attributed to the 
-^-volume of software ivhich'mi/st be developed anii j^ot to inefficiencies in_the 
development of software. 

•.The total software development costs depend on the quantity. of software 
■ which must be developed as well as the efficiency in developing software. 
The increased software costs prdjfected for the future result from the fact 
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that with widespread use of hardware there will be a greater quantity of 
software required for the additional systems. Tending to offset this? increase 
In the cost of pf^oviding more software will be a^decllne in the average cost 
^:F^ developing the new software packages. This' decline In average*^cost will 
be due to the development of more sophisticated high level prograinml ng lan- 
guages. .'These programming languages'will be highly user-oriented and wjll 
reduce labor riequirements for developing specialized applications programs. 
The total costs of developing and maintaining the high level programming 
languages will be high,, but^these costs cjan be anwrtlzed over a large number 
of applications which will tend to drive doWn the average cost. 

Almost every poteQtfal application of computer technology within the 
Navy will involve changes in training programs. The extent of cljanges which 
are economically efficient is one fundamental question which an economic 
analysis must answer. A rational analysii would Involve a stu(^y of each com- 
ponent of the existing training program to determine whict), component or module 
can be efficiently computerized. Since we seldom start from *'g|f*ound zero/* 
there is a need to evaluate the existing capital resource base and determine 
the most efficient means of integrating the new computer technology into the 
existing system. * The total replacement of sunk capital will seldom be econom- 
ically efficient, ^specially in the-short run* However^,^ In the long-run it 
may very well b^^more efficient to plan for total replacement. Each problem 
requires a specific analVsis, and that analysis must be based on an incremental 
approach: 

The efficienqy of each candidate system in,,^eting the various management 
and delivery functions will undoubtedly differ. would be unusual to,find^a 
single computerized system which would be most efficient in carrying out all 
•management aod delivery functions. The computer system' ultimately selected 
for each application must be one which results from a melding of the various 
performance requirements Vith the most cost-effective computer system for 
performing the composite ofJ^those functions. \. 
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SECTION III , \ ' ^ • 

INSTRUCTIONAL MANAGEMENT SYSTEMS * . 

INSTRUCTIONAL MANAGEMENT SYSTEM MODEL* 

A general model 1s presented here relating general management functions 
to' instructional management functions. It shows that the instructional 
management process is not unique, and 1t 1s suggested that established manage- 
ment techniques, when properly ^applied to instruction, can result in greater 
management efficiency. 

' ' * * " ^ -. 

, In spite of the fact that state-of-the-art ihstructional management 

o 

systems are structured in a variety of ways, they nevertheless exhibit common 
mnagement characteristics. These characteristics ^include planning, organizing 
cormiand/executing, coordinating, and controlling (George, 1972). A-general 
management modeKwIth these functional elements Is represented diagrammatlcally 
with the addition of^an evaluation function and feedback loop in, figure 7.^ 
Without a feedback path, the system would operate "open loop" and the output 
product gould deviate significantly from plan. 

With this general model, it Is possible 'to relate general management to 
instructional management. This relationship is presented diagrarmiatically in__ 
figure 8. For the instructional management case, the task becomes one of 
managing a .training system in which the learning proems consists of developing 
' skills, knowledges, and attitudes. Entry testing provides a student inp'ut ^ 
measure; progress testing allows for progress evaluation and prescriptive 
guidaftce during the learning process; and output testing certifies that 
training objectives have been satisfiedr This model can be used ^tb compare 
specif!^ jjfi^pes bf Instructional management systems described in subsequent ^ 
paragraorf^ in terms of the degree and efficiency with which they are capable 
of accdvlishing the identified. function. It Is, however, not meant to be a 
comprehensi ve model of the instructional management process. 
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• Host "current ilistructional. management systems can be categoHzed-;as 



instructor-based instructional management or compyter.-based instructional 



management* — The-fnstructor^b^sedxase relates 'to traditional methods of 

instructionsil management without benefit of computer suj^port. The computer- 
based cage relates to computer- supported instructional management qnd admipis- 
' ■ trative data handling processes. Th^ distinchons between these two generic 
- C^tegoHes are blurred by many possible types of application of the computer,, 
ifi the. instructional process. For example, an instructor-managed system'ln 
wtiich student training and administrative data are stored in and. processed by 
a computer might be termed a form "of computer-based management. However, in 
thfs study, the term computer-based instructional management is restricted. to 
computer management of instructional processes and not to computerized 
—administrative, supportrof deiTver7"prdcesses.. The primary i-eason for this 
^ differentiation is that many existing CMI systems provide administrative , ^ 
information processing and direct computer-based delivery of instruction. 
These systems would properly fall withiji the category of computer-based 
'instructional systems but not totally within ttie subcategory of computer- 
based instructional management systems. 

^ In-this study, the concept of computer-based instructional management 
willapply to a range of computer supported functions. The minimal case is 
the use of a computer for test scoring. Tfiis would be similar to the instructor- 
based m^na§ement cas& with the s..igle exception of computerized test scoring. 
At the opposite end of the computer-ba,sed^nstructional management spectrum _ 
would be the hypothetical "training factory" (Baker 1978) in which all training^ 
management functions are computerized. In assessing the dozen or more instruc- 
tional management systems which are currently available, or any other alterna- 
tive which might be designed, several key issues must be addressed to insure 
successful implementation: (1) Are they applicable in all training situations? 

■ (2) Are they now or are they projected to be economically efficijent/effective? 
(3) Are they operatfonally and logistically supportable over their l*fe 
cycle? (4) To what extent are they subject to technological or functional 

' obsolescence? (5)^ Do they have top management support? (6) Is the user 
community comnitted tcT'the implementation and ma.ntennnce of the system? (7) 
Is funding support assured for the life cycle of the system? Technological 

Er|c 29, 
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7 considerations, which are well within the state-of-the-art, do not appear to 

T[ be a constraining factor for the 1980 to 1995 time frame. 

^^"CSittSYSTErTmRPgARE flND SOFTWArTARCHITECT URE 

TraditionallyV computer-based' instructional systems have consisted of 
^. - one .or mor^ centra] processors coinmunicating' with a number of' time-sharing 
. terminals. This arrangement provides an interactive input-output capability 
which has been applifed for instructional management, instructional delivery, 
and administrative data'handling functions. For the typical case, this 
^ arrangement is still considered to be -a relatively direct arid effective means 

of satisfying computer-based instructional system. requirements. However, as 
-^^-application. of"^Offlpater-ba"sea-iT)structiDnal technology becomes more 
widespread and as computer network architecture technology and capabilities 
^ . evolve, additional longer rahge considerations 'become apparent. It should 
first be determined if thei application of computer-based instructional tech- 
nology in Navy training is likely to increase during the long-range planning 
period and, if so, at. what rate. The results of this determination should 
then b^ used to project the sequencing of integrating CBI functions into 
eJtistirjg training systems. When this instructional technology need/capability/ 
implemeintation projection has been completed, system architectural alternatives 
can be fJeveloped. For the case oi' Navy training, it is envisioned that "the 
• developijient and implementation of a Navy-wide computer-bas^d instructional 
management systati would be more extensive than any computer-based training 
application currently in tise'. 8ased upon an analysis of state-of-the-art ' 
systems^ the following system characteristics hJive been identified as develop- 
ment gu-^delines: ' 

1. System hardware architecture should not reduce procurement options 
to a single supplier for processing, storage, or communications components. 

2. ! Softvare architectural design should follow state-of-the»art 
thinking in relation to usor oriented high level language development, 

, structured programming,- program modularization, tustdm operating systems,, 
documentation relevancy, and system wide software compatibility. 

ERLC i i ^^30 
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3. ".Distributed network architecture should provide a data base design 
"^icb allows glooai netvrorfc data .access only, where required. Local data base 

"control sho ul d pe the general irui e wht^vvvi^r pir^rHpaMa ^,,^444^4^^ '■ 

standards for coniminic atloii protocgl^djLtaJhandljnQ^ 
and data base management should be developed to preclude a dependence on 
single supplier system;;. ^ It is. highly probable tbat computer-based instruct 
tional management systems w?ll eventually interface with large numbers of 
unique processors {intelligent terminals, special training devices, CAI 
systems, video graphic terminals, and videodisc controllers). System comnun- 
ications standards will be essential if interface, compatibility is to be 
achieved. ^ . 



4. -Computer software management systems together with effective con-* 
figuration^control systems are essential for any large scale computer-based ' 
. development. Department of Defense (pOD) software management systems have 
been evolving for a number of years and are now reasonably effective although 
costly to implement and maintain. Because these systems are, in essence, 
management control systems, th^ should be carefully evaluated in terms of 
.their ability to maintain currency in an environment of continuous change. 
Since a computer-based system of this type becomes a management tool, it Is 
important that control be exercised by training program management. 

In assessl^ig current state-of-the-art systems, it appears that the issue 
again becomes one of implementation considerations. The hardware and soft- 
ware are available today to mechanize/automate almost any falling function 
envisioned. . Computer cost Is decreasing and manpower cost'is increasing, 
both a*t rates which are" likely to bring about wide application of computer- 
based training during the next decade. Consequently, it will be necessary to 
provide the needed planning and- operational support to assure operational 
effectiveness in, the desired time frame.. 
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STATE-OF*THE-ART CBIM SYSTEMS 



ihenap^RmWof CBTm. often referred to as wter managed irfstruc-' 
-t'ion. (CMl-K-has growirsteadily during the past decade.- fen private and' 
public sector applications were identified and analyzed in 1974 (Middleton/'. 
Papetti, and Micheli. 1974). Although growth in the number of systems has 
not increased significantly since then, student loading has continued to ' 
Increase and system characteristics have continued to evolve. The Navy 
training CMI systetrt is an example of a large-scale centralized (nationwide 
central processing), system which was well conceived and which continues to 
perfonn effectively. rhe.Air Force Advanced Instructional System (aIS) is an 

example bf a decentralized (base level autonomous oper^tiorO-^system.-which 

like the Versatile Training System (VTS). the Aviation Training Support 
System (ATSS), and the Amy Automated Instructional Management System (AIMS) 
appears to have many of the necessary CBIM characteristics. However, training 
center or school-based systems, sucfi as these requ_i_re extensive instructional 
program development, computer progra|iniing, and configuration management 
support. T^se. support requirements, combined with significant front-end 
investment. Jfaver caused some extension in the implementation schedules of the 
above systems. The long-term payoff still appears on the horizon, but short- 
run expansion schedules are not expected to meet original goals. 



• State-of-the-art military CMI syst^s have- been well described in a 
number of studies (Orlansky and String. T979; Zajkowski. et al.. 1979; 
. Middleton. et al.. T974) and therefore are not descHbed in detiil in "this 
report. * , 

# CBIM TRENDS . * ■ ^ " , 

During the course of this study, many vajuable concepts relating to 
.future computer applications iti instructional^management „ere offered by 
experts in the field. This expert opinion and the examination of related 

■^pJ?er L1ed1n^f'j;..jHr*^^J''''! *° developments .ii, com- 
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research led to the development of two major conclusions. THp.p .^J.u ,^_ont 

^ _ 



- 1. There is a high probability that the use of CBIM will increase- 
■ significantly during the 1985 to 1995 time frame. The trends identified 
which support this* position are: , \ 

a. decreasing computer system hardware costs - - 

b. . increased software development efficiency 

• c. .Increased capability of applications software 
. - d. wider; user acceptance of computer-based applications. 

2. During the 1985 to 1995 time frame, CBIM will become decentralized 
and highly autonomous. . However, strict adherence to centra] management 
^ policy and network standards will be essential for a.global computer inter- 
. coimunications. capability which appears to be a necessary goal for system* 
^ manageinent and control. The trends identified which support this conclusion 
are: 

a. Small, powerful, computer systems will be available at low cost 
for remote stand-alone operation. . ■ , 

b. Reliability, maintainability, supportability, and versatility 
figures of merit for stand-alone computer~systems continue to improve. 

c. New network architectures which allow for distributed data - 
base manag^ent, remote processing, and central network control are continu- 
ing to evol-ve. ' 

d. -A strong trend toward stand-alone CBID, as evidenced by current 
CAI and device based'training applications, will bring about a requirement 
for interfacing these delivery systems with instructional management systems. 
This can be accomplished most easily with a highly decentralized CBIM capa- 
bility under control of a global network manager. 

r 

- n ^ 
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. ■ . INSTRUCTIONAL DELIVERY SYSTEMS . : 

This section defines key terms iii instructional delivery, surimarizes 
major categories of instructional me(iia. provides a brief analysis of selected 
media/delivery systems, arid identifies trends in media development. In this 
report the term "media" will refer to the means for presenting instructional 
material to learners; for example, b^oks. audiotapes, and filmstrips-^(see ' 
CNET Instruction 1500. 12>. Tiie term "instructional delivery system'^ will refer 

• to all of the elements- with which the student must interact during the learn- 
ing process to achieve instructional goals'. The structure of this delivery 

. system determines to .a major extent the organization , and presentation of 
infomation pertinent to training, in other words, an instructional delivery 
system is composed of a student, a method (or instructional strategy)! and ' 

^ spnte form of media. One of the-sfmplest instructional delivery systems might" 
consist of^a student and printed instructional materials, while one of the ^ 
most complex might combine the student with an interactive adaptive CAI • 
system. 

Factors that must be' considered in selecting media for the instructional 
delivery system should include the nature of the tasks, learning strategies, 
media types available, costs of alternative media mixes, state of development 
of proposed media approaches, and resources require(| for courseware develop- 
ment. TA5G's Training Effectiveness and Cost Effecti'veness Prediction (TECEP) 
technique (Braby. 1975) is a procedure for chooSirig instructional delivery 
systems, it i.s usefuTin defining strategies for meeting training objectives, 
choosingr instructional delivery systems for carrying out these strategies, 
.and identifying the relative cost of the alternatives. However, experience 
with the TECEP technique has shown that choosing an optimum instructional 
delivery system for milit^iry, training objectives is a subtle and complex < * 
decision making task that cannot be completely procedural ized. Furthfermore. 
users of the TECEP technique must possess expert knowledge of media. The use 
of, TECEP» however, should result in choosing the most cost-effective ijistruc- *" 
tional delivery system. 

. " 27 ■ 
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CATEGORIES' OF INSTRUCTIONAL MEDIA 



. Braby {1975» table 16) lists a "Media Pool". of 89 types of instructional - 
fnedia organized into seven categories. These categories are print material,, 
audio-only systems^ visual-only systems^ audio-visual systems, CAI/CMI» simu- 
lated .arid" operational systtemst and special or nonstaiidard items. They.are^ the 
major forms of instrucfifilial media beirv^ used or^being considered for use in 
military training systems." Examples from each caiiegary are shown in table 2. 
Another categorization of instructional media i$ shown in table 3. ' . . 

As a matter of interest, table 4 is included^to show how 42 Am(y instruc- 
tional media were grouped into 12 "families" (SilW, Benniclc» Butler, and 
Benesctv->978). 1 " 

Oft ^ . ^ \ ■' ' ^ 

Videodisc and 2^0 interactive* computer grapjjic display systems have a high 
potential for innovative applicatipn in training^ They also have a great deal 
of p6w,er in information transfer technology its well as possessing the possi- 
bility^for increased achievanent and transfer of training. These systems will 
besliscussed in more detail in later sections of this report; 

STATEiDF-THE-ART OR -INSTRUCTIONAL DElIVERlf SYSTEMS 



IilJF-THE^ 



The following paragraphs discuss available media in terms of their capa- 
bility to improve instructidnaKefficiency* 

' ■ ■ r / ^ J . ' ■ 

To effectively design an instructional system^ the training system 
designer lltust know the capabilities and limitations of various media and their 
related costs. A voluminous and. detailed report by Rhode, Esseff» Pusin» . 
Quirk, and Shulik (1970) has assessed the available conventional media in 
terms of their instructional flexibility, support requirements, and initial 
and operational costs. The reader is referred to this report for detailed 
Wfofmation on specific conventional media. However, several new learning 
fweliia are-deserving, o^some^e?^ attention because of their high potential 
in various aspects of the"^delivery of ^Wstruction* 
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TABLE 2; CATEGORIES OF INSTRUaiONAL MEOIA (Afte^ Braby. 1975) 



Print Materials: 
Audio-Only Systems; 
Visual-Only Systems; 
Audio-Visual Systems: 

CAI/CMI: 

Simulated and Opera- 
tional Systems: 



Special and Non-" 
standard Items; 



books* self-scoring exercises* prograrnmed 
text* study card sets 

audio tape system* dial^ access information 
retrieval system* language laboratory 

microform* panels* mockup* slide projector 
system* teaching machine 

carrel* motion^ picture* teaching machines* 
television (closed circuit* videodisc-and 
vidiBO jtaffe systems) 

PLATO IV* TICCIT " 

on-line and off-line computer sirpalations* 
logic trainers^ operational equipment ^ ' 
synthetically stimulated* simulators 

automatic raters, dry carrel, classroom 



TABLE 3. CATEGORIES OF INSTRUCTIONAL MEOIA (After Logart. 1977) 





Programmed 






ETV . Radio 


Text 


CAI/CMI' 


Audio-'Visual Lecture 


television 


PI ■ 


computer- 


audio ■ ■ , 


video 


programmed 


based 


audio tape 


video tape 


text ^ 


systems * 

1 


auJio recordings 




V 

1 

4 


audio-visual 
films 

.^filmstrip ^ 

motion pictures. 

inulticolor ' ' ' ' 

yisual ' 
-slide 

still pictures/ 

graphs 
tape/slide 
transparencies 
visual 

• 
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tABLE 4; CATEGDtllES OF ARMY INSTRUCTIONAL MEDIA 
(After Silver, et al»,1978) 



"7^ 



-Job-Mdter-lals- 



Field Manuals (FMs) 
Technical Manuals (7Ms) 
•ITDT Technical Manuals 

Printed Materials . : 

TEC Print 

Correspondence Courses 

Training/Combat Literature 

. Soldier's .Manual^Job Book 
SMll Qualification Tests (SQT) 
Amy Training and Evaluation Program "(ARTEP) 

rnsjfcructor with Standard Aids. 

• Class Packets ' ^ ^ 
Charts/Display Boards 
Overhead/T*" ..ibparencies 
, ,Model s/f-kjckups 

Audio-Only 

TEC Audio-Dnly 
Language Labs (GEL) 

Audio-Visual ■ ^ 

TEC Audio/Visual (BESELER CUE/SEE) 
Slides/Sound Slides 
Arnvy Training Films 

Television/Video Recording "v, 

^ f 
Classroom Closed-QiroTitJ Television (CCTV) 

Television Trainer" (TyT)',' 

Video-Disc ■ ^ 

Computer-Assisted/Manag^d Instruction (CAI/CMI) ' 

, ' . ■ r . ■ 

. Remote-Access PLANIT 
PUTO IVAUTOR ■ . 

ABASCUSiComputerlzeil Training System (CTS) 



3D 



37 ^ 



TAEG Report No. 89 



TABLE 4. CATEGORIES OF ARMY -INSTRUCTIONAL MEOIA (continued) 
(After Silver, et al., 1978) 



Embedded Tjrainim (ET)' 

. Operational TACFIRE PLANIT 

TACFIRF Training System (TTS) 
■ TACFIfi"E;Subsystem Team Training (TSTT) 

Training iDevices/Simulators 

ft ' ' ' 

Sand Table 

Fire Control Simulator BT-33 
Observed Fiice, Trainer (OFT) 
Artillery Oirect Fire Trainer (AOFT) . 
M-31 Field Artillery Trainer 

Tactical Engagement Simulations (TES) , 

SCOPES ^ 

REAItRAIN (Exercise for Contiined Arms Elements) 
MILES ■ ~, 

Command/ Staff Battle Simulations ' 

Tactical Exercise Without Troops "(TEWTS) 
CPX Simulation Facility . ' • . 

-CAhWS (Computer Assisted, Map Maneuver SystSem) 
Combined Arms Taetical Training Simulator (CATTS) 
FIREFIGHT " ^ 

DUNN KEMPF ' ' ■ ' ' 

PEGASUS ' ! 

First Battle' ' 
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The current typical Navy delivery system is based primarily on print 
(nedia moderated, and managed by instructors (p^us some usf" of conventional 
audiovisual media). This approach appears to have reached'its limit of appli- 
caitions effectiveness. Not only are marginal efficiency improvements very 
small but usage, Is mismatched to the current training need (Buhderson» 1979a}^ 

Just as conventional audiovisual media have been integrated into the 
instructional system by matching , them and their specific advantages wfth th^ 
objectives of training, it is now essential- to accomplish this for the newer 
nedia; e.g. »*turrent CAI systems and' the other more sophisticated audio^-visual 
inedia such as videodiscs^ Their ^successful use requires analysis of their , 
strengths and limitations through continuing research. For example, after 20 
years of CAlJsystem research it has been established that the primary benefit 
of these sysjtems is derived from their capability for individualizing instruc- 
tion. ^ it is capable of adjusting instruction to meet student performance. 
For the current status of CAI, CHI, IMlJ and PI,. see^Orlansky and String 
(1979) and Zajkowski, et al. (1979). Mpresentati ve of the data to be found 
in these reports are tables 5 and 6, r&yroduced here to show the effectiveness 
of CAI and the extent to whicOtS^^ing used operationally in Navy 'training. 
The roots of computer applications for the 1980's. exist now, and there is not 
mch likelihood that new design concepts will change how they are applied. 
Chaf(|es will most likely be ^limited to scale of use and iii who will^ be using 
them (Office of Technology Assessment;, 1979). , ^ ' 



Two relatively new forms of instructional media have great potential for 
future instructional delivery systems*, namely, 2-D interactive computer 
graphic display systems and videodisc. 2-D interactive computer graphic 
displays pVobably have more potential for wide application during the 1985 to' 
1995 time frame than ariy other CBID medium. This delivery mode can provide 
visual 2-D simulations with keyboard, light pen, sonic pen, photo sensing or 
touch' panel student interactive capability. This interactive feature can be 
used in combination with direct input .3-D training compoifents and other 
peripheral equipment for a'fully integrated training system capability. 
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TABLE ^. SUMMARY OF FINDINGS ON CAI AND'CMI, COMPARED 

TO CONVENTIONAL INSTR^]CTIO^f ' * . 
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Source: Orlansky and String (1979) 
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TABLE 6. COMPUTER-AIDED INSTRUCTIONAL PROGRAMS .P!.A"N- 'IN PLACE IN NAVY TRAINING 



> 

CAI IdMtlMcttloa 


Training Use 


0 

HiMfaer or StudtnU 
Stations 


Locations 


> 

flmrks 


TICCIT (HITftE/Hiiftltlnft^ 




^ Undetenilned 


north Isla»d 
Cecil F^ld 


1A Use 


CO?/3 (EducttloMi 
CoipuUr Corpor4t1oo*s 

^- Aiul 'l*J*<»*ii€ 4 Ami ' 


AS 

tUrlne 
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91 
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60 
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S systems 
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GETS (Geneni Clectrk^ ' 

conUlned. Intenctlve 
trilnlng console) 


TRIDENT (stntesk 
Weapons Training) 

TRfDENT (Engjneerlng 
Operttlons Training) * 


■ r 


Banaor 
Bangor 


1 




OS '■A" Schooi 


60 


Uindeter«tned 


NTDS Training 

FY 84 IiHpltMntatlon 


-C1(0} CAI^'sten 


CT "A*" School. 


".15^ 


Undetermined 


Basic Cowunlcatlons and 
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Because of the computer dynamic capability for changing graphic display 
pardmetiers (e.g., voltage or resistance values In electronics training), 
performance-bAsed training will become possible. This Is currently being 
accomplished very effectively on such systems as TICCIT, PLATO, and GETS, but 
the cost Is still too high to justify wide scale application. However, the 
success of these systems along with projections of decreasing costs* makes 
this type of delivery very promising for future applications. Perfo;?r»nc'e- 
based training and testing appears to be a major design goal for future 
training systems. 

Videodisc Is considered a rei^olutionary develppment .1h t/alning tech- 
nology because of Its, mass Information storage capabilities and the flexibil- 
ity in media'materials vrfiich can be; stored. It can store all displays that 
are now delivered on airy audiovisual medium, such as color slides, super 8 mm 
film, videotape, or motion pictjJre. . The 108,000 video pages -of a single disc 1 
can be used to deliver information. in a small fraction of the space required | 
by print documentation. It Integrates the features of several different \ 

media into one simple storage and retrieval system. However, the authoring I 
■■ ■ . ■ ■ s 

and production of one single master videotape of such materials prior to 

making master vltleodiscs also combines the problems of these meflia (Bun^erson | 

. and Campbell, 1979).' > . .\ ' . j 

Bundersbn, Jui*v1s, and Mendenhall (1979) studied the implications for 
the videodisc authoring process of the following eight areas: the delivery 
system used (manually^controlled^ microprocessor controlled, computer con- 

tr„oJ3ed_with_ei.ther-4-CAI or TV display)^ media selection during ISD; instruct. 

' tional strategies; author mock-up and simulation pripr to premastering; pre- 
mastiering; mastering and replication;' composition of videodisc authoring < 

. team;, and evaluation.- These researchers concluded that although optical 
videodisc technology and its associated authoring technologies can be expected 1 

to be undergoing. change during the next few years, videodisc has significant 

' 'I 
potential as a new media delivery ^ystem alternative for the Navy. However* 

they reconmend that the Navy should.not.plan to use videodiscs in'^the .itmte- ■ 

(<^^te future but that the Naval Education ,^nd Training Comnand (NAVEDTRACOM) \ 

(Ould track the development of the field. They further recommended that the 
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, , Navy should conduct RSD involving the development, and. (nastering of^a series 
of videodisc strategies on promising delivery system alternatives. 

* ^ r 

IF ■ ^ 

It H intsresting to note that Fox ^(1980) believes that only limited 
resear'ch -is needed. He believes it is possible to demonstrate new approaches 
for instructional management and, delivery systems by app^lying state-of-the- 
art instructional technology in existing classroom settings. Fox^^^T^Thas 
developed a microprocessor-driven Visual Image CojitiH>H63^Vis7 I . Con) pro*- 
jector. His stand-alone instruction. delivery system presents instructional 
^ materials on a functionally equivalent basis to that which can be accomplished 
on a video system. When the instructional profiram is validated .on the Vis. I. 
.Con, it can be transferred to a videodisc, which can be controlled by a 
microprocessor. This microprocessor can then function as a master controller 
for both instructional management and instructional delivery. Fox has seen 
no serious problems associated with current videodisc applications and 
'^believes it is now ready for use as an operational component of new instruc- 
tional systems. 

TRENDS 

"Anything in the mat-ket place in 1988 is alre^^y in the laboratory in 
. . 1978" (Fields, 1979). 'The "building blocks" are bel-^eved to* be known. - What 
will be done in the future is a combining of the "bloc|cs ' into systons and 
\ applications. 

According to the Ofhce of Technology Assessment (1979), "the. roots of 
most new computer applications in the 1980's exist now, and there is not much 
' - likelihood that a new concept of computer design will change the way we use 
, them. Changes will 'be in scale of use and in who will be using them." 

- * ^ I f 

^ ■ ' . 

Bunderson (1979b) states that it is not possible for DOD to maintain 

^conventional methods oriented around stand-up instructors and printed materials 

.since training, budgets are being cut every year. New courseware forms and 

new delivery systems must emerge. These new systems will be dependent on the 

^ Widespread distribution of computer based delivery systems/ 
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In thfe early 1980's 'small personal c(Mnputers w^tfMisited-fflaiRJfy^^ tie 
available to provide answer analysisanijfeedbacR7prwnpting help, and diagr 
nostics. The videodiseJd^MifTjsed increasingly as a medium for ^distribution 
of instjcuct+on'; ft will becwne competitive with other delivery systems such 
as audiocassettes, vfdeocassettes, floppy discs, .and printed pages. Print and 
audiovisual media will be transferred onto videodiscs and microcomputer-based 
systems. It will' be cost effective to transfer existing courseware products 
that ha^e been validated and are widely used to a single audiovisual delivery 
system such as a videodisc player. 

The' cost of microprocessor and memory module conq}onents which p^rovide the 
computer capabi? ity for graphic display systems will continue to decrease in 
price for the next decade. Consequently, large scale application of 2-D 
interactive computer graphic display systems will be limited primarily by 
software development cost. Even in this relatively high cost area, many 
software advances are being made for entering and storing graphic Information 
with low cost direct input devices. It is probable, that within 10 years, 
computer storage of graphic Information will be the lowest cost option avail*^ 
able for storing training-related visual information. 

CAI will become mov^e comrnon as co«nputer-^based delivery systems becoine 
more widely used. The trend will be away from linear or fixed variety media 
which just present information. The trend will be to provide interactions,, 
between the trainee and media. ^ , , ^ ' 

, In addition, intelligent courseware, or intelligent CAI (ICAI) will 
increase rapidjy. Strategies such as the following will be used: simulation 
of a performance environment (e.g., troubleshooting a circuit}^, diagnostic " " 
model of the trainee's status (e.g., discrepancies in procedural knowledge), 
prescriptive model for tutoring a trainee (e.g., giving hints for trouble- , 
shooting) (Bunderso'n, 1979b). 

In the late 1980's and beyond,- total distributed instructional delivery 
systems will be available. The system will he stand-alone with, intelligent 
terminals. The learning station will have tha storage and display character- 

wit * • "• 
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Istlcs of videodiscs so altl audiovisual courseware will be displayed by a 
single medium; CAI will be the preferred mode for providing training with 
feedback'at a job site or on-the-job training, which, according to^canland' 
J[ig78), will Inci^ase due to the severe reductions In EDTRACOM resources,, 

^ At some duty stations. Instructors may not be available, .whereas In 
schoolhouses they will still be essential, though their roles will change. 
They will function In ways, computerized systems cannot; e,g,, providing models 
of the kind of sailor desired, stimulating enthusiasm and. motivation, diag- 
nosing problems not Included In the computerized system, and coordinating 
tasks that Integrati^ skills learned with the computer. 
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"* V. 

SECTION V. • 

INSTRUCTIONAL'MAtiAGEMENT ALTERNATIVES . ' .. 

- ,\ " ' 
This section describes instructional management alternatives which are 

.de^d to- be v.iable in the 1985-1995- time frame and provides an initial assess- 
ment of key management and economic characterist'0:s for each. Two general 
categories of instructional management have been identified; namely* Instructor 
Based Instructional Management (IBIM) and'Computer Based Instructional Hanage- 

,ment (CBIMJ. Each of these major categories can' be further refined- into^sub- 
categories by the inclusion of various combinatio'n^s of the features of (1) 
centralized' processing and data base control vs. decentralized processing and 
data base cont'^ol and (2), the use of existing technology vs. new technology 
(state-of-the-art hartfware and software). The eight possible combinations of, 
these features are shown in table 7. ;However, only four of these eight are 

. considered viable in the 1985-1995. time frame. These are; 

1. Instructor Based Instructional Management (IBIMj^—descriptive of 
traditional Navy instruction. ■ 

2. Centralized C^IH using existing technology^-descriptive of the 
existing Navy CM I system. 

•3. Decentralized CBIM using existing' technologiy—descriptive .of the 
Navy's VTS and ATSS and the Air Force's AIS. 

4. Decentralized CBIM using new technology. This is a reasonable ■ 
description. for a 1985 to 1995 system which would most likely be developed 
under an RDTSE. program. ^ 

' Each alternaj^ive identified. in table 7 should be considered to represent- 
a class of alternatives. ' The addition of some form of CBIA, CBIS, and CBID 
to each/of these major classes is required to define a specific, computer based 
training application. It- is the latter completely specified system that will 
provide the basis for economic comparisons of various systems. Until such 

■39 

4? . ^ 



TABLE 7, INSTRUCTIONAL MANAGEMENT ALTERNATIVES., 



INSTRUCTIONAL 
MANAGEMENT 
METHOD 



EXISTING TECHNOLOGY 



NEW TECHNOLOGY 



CENTRALIZED DECENTRALIZED 



CENTRALIZED 



DECENTRALIZED 



IBIM 



CURRENT 

im. 



.CBIM 



CURRENT 




ATSS 




■N& 




TYPE 


X 


•DEVELOPMENT 


. CMI , 




SYSTEM 




(RDT&E) 



NOTES: 



IBIM - Instructor BaSed Instructional Management 
CBIM - computer Based Instructional Management ^ 



Viable Alternative 



X - Nonviable Alternative 
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complete specifications are developed only the most rudimentary assessment of 
the four major classes of alternatives is possible. * 

Table 8 serves to, identify system features and to summarize the ofJtcome 
of an initial assessment of each of the four classes of alternatives based on^ 
tftese features. £ach of , the-evaluative entries in the cells of the table 
represents a consensus of the study team based on data collected during the 

study. 'The data for alternatives V and 2 constitute an assessment of current 

* 

capability while the data for alternatives 3 and 4 constitute predictions of 
the system capabilities. 

Note that the major characteristics are divided into two major categories-' 
management and econonic. In the fOTiner category all management characteristics 
are considered to be essential .in the 1985 to 1995 framework and therefore " 
should serve as the basis for a comparative analysis.. In the economic charac-' 
ter1st1cs» Investment and operating costs, which form the basis for'life cycle 
cost» combined with training effectiveness provide an estimate of life cycle 
efficiency. .These are the primary -economic characteristics which form a basis 
^for a comparative analysis. Expansion cost and efficiency refer to increasing 
the system capability beyond present or "designed capabilities. 
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TABU 8. CHARACTERISTICS OF ALTERNATIVES 



ALTEANkTlVE 


CHARACTERISTICS 


HAN/iGEFtfHT 


ECONOMIC " 


Info 

Acctssl- 
blll^ 


Objec- 
tives ' 
and Test 
Item Data 
Sank 


Resource 

Manage* 

fnent 

Control 

System 


Student 
. Sched* 
uling 
Systai 


Report- 
ing. 
System 


Manage-" 
lient 
Cofrtrol 
'Systtm 


Invest* 

jnent 

Cost 


Opera* 
ting 
Cost - 


Train- 
ing 

Etfec* 
^^tlve* 
ness 


Life 
Cycle 
Effi- 
ciency 


Expan* 

slon 

Cost 


' Expan- 

. slon 
Effi- 
ciency 




tiectntratfied IBIK 

- . (WD ' . 


1 


Z 


2 


2 


^ 2 


3 \ 


low 


- Riedlum 


'tow 


medium 
■■■ 


medium 


medium 


Central f zed CSIH 
ttchnolgjf) 


. 3 


4 


2 ^ 


< 

4 . 


s 3 


3 


low 


* 

* inedlutrt 


medium 


high 


high 




Decentralized CBIH 
' (existing 
technology) 






4 


4 ' 


4 ( 


i 


medium 


' nedluii 


medium 


^h^gh" 


medium 


. high \ 


Occen^rillzed C61K 
(new tecttnology) 


5 


5 


5 


5 


5 


5 


high 


medium 


..high 


medium 

ft 


medium 


high 



'TO 
TO 

o 
o 



S - Blgtr . . 
4 * Above Average 
3 " Average 
2 " Below Average 
l-LoN 
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. . ' SECTIOfi VI 

. RECOMMENDATIONS IN SUPPORT Of 1985 TO 1995 

CBI. SYSTEM DEVELOPMENT ■ ^ * 

Reconmendatlons for support of CBI system development are prjesented 
below; They are based on study findings and assume that a new distributed 
ccHnputer system app,roach will be recognized. as the most viable for future 
ppllcatlons. ; 

■ ■ # ' • ■ ' 

■ vl. Establish a CNET staff management group'with single-point authority 
and responsibility for all ciwnputer based Instructional programs. 

' 2. Establish a management and economic analysis comnilttee with member- 
ship from CNET, TAEG, 'and funqtlonals to develop analysis procedures, review 
proposals, dnd Insure valid economic assessment of all proposed CBI appllca^ 
tlons* This committee should be supportive of the CBI staff management group 
yet independent, for purposes of m^ffagement and economic ana1ys,is* \ 




3. Cpndiuct an economic analysis to assess life-cytle cost-benefit for 
each of the four CBI alternatives proposecl^. 

^ 4* Deter ine the viable computer network architectural designs for 
distributed procfessing/distributed data base» communications protocol » and 
network ^management for proposed systems. s 

5* Prepare an Operational Requirement (OR) to identify viable strategies 
for integrating computer based Instructional delivery systems into existing 
Navy training programs dur|ng the 1985 to igg5* time frame* 

6* Prepare an OR to determine the types of software development programs 
needed to provide the necessary software sypport for Navy CBI applications 
during the 1§85 to igg5timc frame* 
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7; Determine life-cycle support requirements for large scale CBI 
applications during the. 1985 to 1.995* time frame'. 



■ _ 8. Prepare' a master pUn fqr the continued utilization and support of 
the existing Navy CHI system during development and integration of the 1985 • 
to 3995 time frame replacement system. ' - ■ . » 
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ERIC 



53 62 



TAEG Report No- '89 

rfAJOR'cATEGORIES, SUBCATEGORI£S AND FUNCTIONS OF CBI' 

' ' ' ..... 

COWUTER BASED INSTRUCTIONAL MANAGEMENT (CBIM) ' ' 

m 

V. Student Scheduling/Sequencing . This functional category provides 
for the control and tracking of , all student activity beginning with registra- 
tion and ending with course completion or ^p.reccHnpletidn termination. It 
utilizes information from student records* entry level test resuTts», course 
4>t9ani2atiort amd sequencing strategy/' progress and diagnostic test results* - 
administrative inputs* student location control reports* , time to complete 
estimates* and- time in process measures as as^ , .i6nt alternatives, are con- 
tinuously assessed for optimal scheduling. . Student assignment*^ scheduling* 
and tracking functions'include: * ' 

^ a. Student Registration provides for the initialization^of the 

student into the training, system. It consists of entering student informa- 
tion into the training system data files to allow for the maintenance of 
student assignment* progress* and history records. It also provides student ^ 
profile information for assessment of viable training track alternatives and 
/ for estimates of course completion. 

\ ^ - • 

b. Student assignment and scheduling includes the functions 
required to analyze current student status jand to generate student learning 
guides. Student^progress' indices, course organization and mapping*. learning 
.resource availability* and progress test response patterns are analyzed to 
provide^ a basts for generating these guides which, specify a training path. 
This proSess relates student assignment options to student status and learn- 
ing resource availability and then selects a training path which .best satisfies 
the training objectives. ^ - * 

c. student tracking enters the scheduled, activity into the student^ 
schedulifjg and tracking program as an in-process activity when a training path 
is selected.. This activity is then tracked and is utilized as a basis for 
measuring student progress. - ' * ^ 
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2. Hodule-Usson Data Bank . The training system module and lesson 
data bank VK>uld consist of a dedicated storage area .within the training 
center computerized cpmnon data ba^e which would contain descriptive and 
identification information for learning modules and lessonvwithin the train- 
ing center course organization hierarchyir' In addition, it wuld contain a 
description and identification of the training materials, training objectives 
progress and. diagnostic test items and audit vectors for instructional 
accountability. The functions within this categ6ry include: 

a. Training objectives files would fully describe an* identify 
the training objectives related to course modules and lessons. These, train- 
ing Objectives would nornially be developed during a front end analysis for 
the Instructional Program Development^q^se. However* objectives could also 
be developed for existing programs of instruction through the analysis of 
course content. \ 

b. Training material description and identification records would 
be developed for the"^ training material used in support of module or lesson 
.level instruction. This training material identifi coition system would be 
-used for student guide generation related to specific module and lesson 

assignment. . - . ■ ■ 

c. Test item files would be grouped by module and lesson for 
measuring student achievsnent. These testing materials would also include 
prie. post, and diagnostic test items. ' ^ * > ^ 

d. Course audit and accountability includes al.l of the functions 
required to assure the integrity of learning objectives as they relate to 
learning materials and test items. 

* T 

3. Testing . The training system testing functional category would 
include methods for accessi^ng ajl test items to be presented during the 
testing process^ Test items would relate to the learning material presented 
and would follow predefined testing .strategies. Item generation* response 
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scoring and storage^ item analysis^ diagnostic test item generation^ random'* 
Izatlon of test item sequences* and test security would be Included in this 
category. . ■ - 

a. Test generation would be possible for any module or lesson- 
within a course of instruction. Selection and sequencing of test items would 
be generated in accordance with course 'hierarchy mpping» learning module 
identification* student achievement* and diagnostic test results. Tfests • 
would be generated 1n accordance with the applicable testing strategy. 
Uniqueness resulting from .randomization and customization would also provide 
a basis for developing' the test security system. 

b. Response scoring would apply to the instructor scoring case or 
the machine scoring case in'^which a test key is used as a basis fot^-scoring.- 
However, the computerized case would allow for a decode of test item random- 
zation and would facilitate ati analysis of test response patterns. Conse- 
,quently» greater prescriptive g'uidance flexibility would be possible for the 
cofiffpjjterized case. ~ . ^ 

cl training path selection and prescriptive guidance should be . 
possible for ^ny type of path selection strategy used by tfie instructional 
designer. This function is difficult and time consuming to implement fj^r the 
instructor management case but it is relatively easy to implement for the - 
computerized case. 

d* Test response storage should provide for short term machine 
readab-le storage of'all student test^responses* This test response data bank 
should be, converted tof long term machine readable storage when students' 
coii^)lete training* This data can then serve as a basis for test item apalysi 
and course effectivenes^s evaluation* ' ■ 
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COMPUTER BASED INSTRUCTIONAL ADMINISTRATION (CBIA) 



1. Manaqpagftt and Administration . This WteQory includes all of the 
organizations functions required to adminis^ter instructional programs. It 
is intended to be thel master management contiroT category for al;l other func* J 
^ional categories. The furtctions within this category include the following; 



a. School managagent function Includes the planning, ^ganization, 
and adininistration functions which support school Operations. 

b. Management policy and control includes all of the management 
control mechanisms for assuring compliance with management policy. 'It relates 
to the dissemination of management dire(;xives» to the communications of 
management information for cpntrol structure feedback, and to the measurement 
oif management system effectiveness.^ - - - 

-— c^ PersonneV-manaqemeirt- cons4sts-of-4he~dat^process4^^g-^ 



transactions for th^ a<biiinistrative control of staff and student personnel. 

d. Cotnpiiter systems nanagem^^nt includes all of the organization . 

and management functions required for the acquisition* operation, and maint/- 

L- ' . / 

nance of computer hardware and software. Althoizgh the computer operations 

function wou3d properly ba located within a computer group » all policy, 

standards, and-management control procedures should be^under Jthe, direction of 

,a training *ectorate to assur? compliance wit^j training goals and objectives, 

/ ^ " ^ " ^ . \ 

e* Course organization includes descriptors and\|dentifiers for 

all modules and lessons within a course hierarchy. 

f. ^ Student record control includes all processes related to 
student record entries, information access, and training transactions. 
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g. Data control and cormiunications sut>ports policy making for 
Information standards* integrity, and security. . Although the data managen^nt 
system apd the report generation system components Would be independently 
operable* data control should remain within the training system management 
organization. Administrative information should be cormiunicated from a single 
d^taj^coimw locati_o_nJwiJ:hi_n„.a school. or training center to any external 

organization requiring the information. ^Student data* course data, predictive 

data, and effectiveness data could be. used as a basis for systems analysis, 

"* ■ 

student assignment, and management control.- 

2. Report feneration . This^ func'tional category includes all of the 
programs required to access, convert* organize* ahd^format data and tO' gener- 
ate data output for display or hard copy, it would also include the. genfera- 
tion of unique symbols or graphic displays recjuired for special use applica* 
tionsv The following reports are examples of outputs from t^is' subsystpiq: - 



a. ^ .training sdiedulj&s, 

b. admi ni strati ve, reports 
^ c. computer system status reports ^ 

<k , configuration management Reports 

e. ' course/module/lessoh evaluation reports, 

f. resource management reports \ 

g. , * Instructional Program Developntent status reports 

h. • productivity/efficiency/cost reports ^ j 

i. student learning guides 
j. ' student progress reports 

COMPUTER tfASED INSTRUCTIONAL SUPPORT (CBIS) 

1* ^Training Resource Management . This fgnctPiopal category relates 
to the management of training system resources and should include all per- 
sonnel, fajci^ties, equipment, software, and expendables required during 
executiM Af the tr^^ining process. As a student transitions through a^ 
course of instn .ton, student station assignment, jlearning mate^'ials as^gn- 
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ment» and Instructor consultation will require scj^eduling and control. 
Although this Is currently accomplished in most Navy training settings'by the' 
Instructional staff, there are many case$, in which computer-based systems 
perform these functions.^ ■ ' ? 

■ a. Training resoOrce inventory control includes all of the function$ 
required to- identify, catalog, store, "assign or is$ue, record, and control " 
all/physical jtraining resources. Although personnel resources can be con- 
isidered traiiring resources, they are not included witWn the resource inven* * 
i:ory control function .because their-cpntrol is included in the managenient and 
administration functiSn. A similar situation" exists for facilities and 
facilities support equipment. Although classrooms an^ student, stations are 
scheduled by^resource scheduling programs, facilities mafiagment is not ' 
included as a training management function. \ * " ; ^ , , 



■ ^ b. Training resource scheduling includes the, functional actiyity" 
related ta allocating learning resource? (such a$ student, stations and learn- 



ing materials) in such a way as to maximize the efficiency of resource utilir 
zation and minimize the student waiting time for resource assignment. Sched- 
"^ullDg strategies .shou]d vary in accordance with course 'loading, .resource - 
availability, and student assigranent options. Resource allocation records 
should be-maintaiped for analysis and accountability purposes. 



c. Training resource accountability fuhctions .incljude all of the 
activities relating to the monitoring of the status', utilization, and mainte- 
nance of resources. This function ^also includes record keeping and analysis 
of resource use to pirov.ide a . basis for adjusting resource levels and modifying 
resoupce^stheduling strategies." 

2. Instructioml Program Devgloptnent Coordination . This functional 

category includes all of the activity required to tnaintain schedules for 
. fti"^ ■ ■ " ■ ■ 

lear^ning. mateHal development activity. The training material to be devel- 
oped or under development will normally add to or replace existing training 
resources and consequently will require planning for integration into existing 
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instructional systems* The instructional management/plarv to be utilizfed/in 
conjunction with the new and revised learning mat^nals mgst also be incor- 
porated Into the ^lanagement^ administration, spheduling, and tracking systems. 

at Instructional program development requirements relates tp 
needs assessment and requirements development*. If' the changed or new require- 
ment results in development which will affect a training program, a tracking 
mechanism should be energized during the conceptual phases. In this way» the 
research, develoi^ent, and implementation process will be coordinated-^with . 
the integ/ation of the new or revised training material into the existing ^ 
training program and the necessary planning a^d prograrmiing lead time will be 
provided* ^ ^ i_ 



b. Proilram development schedule functions include the^'planning 
and scheduling activity which precedes ^veldpment* These functions ,would,be 
carried out in coordination with the development age.^t and should result in a 
single development schedule which is tracked by the developers, the imple- , 
menters, the user, and the evaluator* \ ^ 



, ' c* Program development tracking and reporting relates to the 
tracking, activity which takes place durlnij^ the devleopment and inplementation 
process* , It should be accompanied by iiiformation reporting to be used by 
instructional program development managem^t and by instructional process^ 
management* ^ ^ . 

3* Configuration Manag^nqnt . Because training materials and computer 
. programs are constantly in ia state of change for The general case, a need 
^ex'^+^^i to incorporate a cohfiguration management system to track chanties 
? J ijufation status* It is evident that training objectives must relate 

^ing materials, while test items must rjelate to both* What is less 
evident but equally important is that the course scheduling and tracking 
programs and training resource allocation programs must also relate to train- 
ing program objectives. As the process of developing training systems becomes 
more closely aligned with the development of the more, complex weapons systems 
to which they must necessarily relate, a requirement to implement similar 
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and 
systems 



'con;igttration control systems will become evidfent. To design, develop, 

implement some'of today's more^sophisticattd ccxnputer based ^training sy 

iflfithout Incorporating a configuration management system would be to assure 

future degradation and potential system failur^. Configuration management 

techniques have been used effectively for many years on all types of computer 

based syitems and it is not envisioned that the application of these tech- 

! ' / 
niques to computer based training systetns would cause ady significant problems. 

' , \ 'j ' ^ \ ■ / ' 

Some of :he functions which should b6 subject to configuration control include: 



a. 

b. ;, 

c. ' 

ey 

f.' 

9- 



cour'se organization 

fac-ilities "ahdl training stations 



/ 



training equipment and, devices 
module/lessorf .objectives and test items 
learning^materials 
computer haiU^are 

computer Software. / 



Conjfiguration ma na gen en t, would provide the needed change control, status 
accounting, and" system aucit capabilities for continuously" evolving training 
applicatjions. It would allow for operational efficiency under conditions of 
constant' change, a, charactpristie which appears/ essential in today's dynamic 
environment. 



4. i Computer Support. \For tftose functiqnal ca^^gories within the' 
instructijOnal management ^stem which are partially or fully computerized 
there wiljfl be a need to manage\^the computerization process. ^ Both hardwaif*e 
and software systems require di^agnostic progr^ams for preventative and Correc- 
tive maintenance. A full rangevcf maintenance <nd operation support docunen- 
tatiop '^^\id be available for life cycle support. This documentation should 
include sujch typical information as logic' d'lagrams, system block diagrams, 
operating system descriptions, apnlication progratps design and interface 
specifications, data base management organization, and data corrmunications 
requirements and specifications, ml of this support dbcuirentation should be 
controlled within the configuration unanagement functional category. An audit 
capability ihould also exist for validating the performance of any hardware 
or softwareisubsystem. \ ■ 

/ - ■ ■ 
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Computer hardware and software organization should include 
blods^'diagrams and functional descriptions for all hardware inputs process- 

storage and ou^tput subsystems^ and the operating system and applications . 
programs. Data base organization* data base access* ar^ ccflimunications 
protocol specifications should also be provided together with operating 
procedures for system use.^ 

b. Data base organization and manageniant documentation should 
describe file organization* data fomats* data typas* data conversion require- 
luents^ common data pool use* data access* data protection* and report genera-, 
tion interface. Memory maiT^ing for local and shared data should aUo be - 
provided for ease of integrating new programs or modifying existing programs. 
Although much of this information can only be displayed in formats requiring 
interpretation by computer specialists* *an attempt should be made to describe 
all data organization 1p user language and functional block diagrams, , , 

c. Program performance functidnal specifications include all 
documentation describing the functional capabilities of applications programs. 
These functional descriptions provide the basis for validating program per- 
formance and serve as the primary descriptive documents for establishing 
"system design goals. 

d. Interface specifications would maintain a current file for all 
intra-computer and inter-computer hardware and software interface specifica- 
tions. This information would be available in tha form of CRT c ciplay or 
hard copy. output and'^wojld be maintained in accordance with configuration 
manag^n^'t standards. ^ 

e. Data communications protocol would maintain current files on all 
data communications standards. Communications test prograuis and network diag- 
nostic programs would also be Included for initialization and monitoring of 
network 6peralions. 
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f. iystein operating procedures would provide CRT display, hard 
copy printout or operating manual description and identification of all system 
operating procedures. Prorop-ting and menu selection in user teminoldgy would 
be provided for ease of use by a^l operations personnel. 

g . System maintenance support programs and documentation i ncl udes 
prbgraoing for all maintenance suppcrt activity. Maintenance procedures, 
diagnostics, references to maintenance documentation, tools and equipment 
listings, maintenance action record keeping, maintenance analysis reports, 
and other software and hardware maintenance related programs would provide a. 
semi -automated maintenance support capability. 

COMPUTER BASED INSTRUCTIONAL DELIVERY (CBID) 

1. Conventional CAI ■ " ' 

2. Computer Simulation . . . 

3. Computer ControlTed Video . ' 

4. Computer Graphics. . ' 
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